Skip to content

feat(propensity): make a reasoned DECLINE learnable — a third state, with a kind - #45

Merged
stranske merged 2 commits into
mainfrom
claude/declines-and-applies-to
Aug 23, 2026
Merged

feat(propensity): make a reasoned DECLINE learnable — a third state, with a kind#45
stranske merged 2 commits into
mainfrom
claude/declines-and-applies-to

Conversation

@stranske

Copy link
Copy Markdown
Owner

The finding this closes

Three independent audit rounds on 2026-08-23 (Workflows, Trend_Model_Project, Fine-Art-Archive) reached the same conclusion: propensity carried information exactly once, because most candidates sat at the uninformative prior. The missing input was not more consults — it was declines. A capability declined on repo-specific grounds looked byte-identical in the ledger to one nobody ever considered (trig 0, use 0, no 0).

A decline is a THIRD state, not a negative outcome

This is the whole difficulty. A decline means the capability did not run, so recording it as an outcome would bucket it into not_useful — asserting we tried it and it did not help, about something that never executed, corrupting the one signal declines exist to sharpen.

The separation is structural, not conventional: a decline rides on a match event (it genuinely was offered — the only claim the event type makes) tagged source=capability_decline, and usefulness() reads outcome events only. There is no code path from a decline to the posterior. No new event type and no new store: record_promotion already carries a non-match fact on match distinguished by metadata.source.

triggered / declined / not_triggered_silently now partition the candidate set, so "rejected on stated grounds", "offered and ignored" and "never considered" are three different findings. propensity() reports the decline count beside the posterior — "prior, no evidence" and "prior, no evidence, four reasoned rejections" were previously identical readings.

And a decline has a KIND

One undifferentiated count licenses the wrong correction. The third round declined 25 offers across six classes with opposite fixes:

observed kind fix demotable
testgen-lane matched correctly 3× in a read-only audit with no commit target no_landing_zone nothing no
offload declined at 9 surfaces — declared surface-wide, one-subsystem audit scope_too_small a precondition or a narrower declaration yes
frontend-verifier declined on two frontend-less repos, then produced the second-strongest finding of an audit on a repo that does have a display surface precondition_unmet evaluate the condition, do not weaken the binding no
it does not fit this work wrong_match the matcher or the binding yes
held behind a default-OFF switch or shadow status gated_off the gate, on its own evidence no
wanted, not affordable this run deferred nothing no

demotable is a property of the kind, declared once in DECLINE_KINDS. An unknown kind is refused rather than coerced — a typo silently becoming unspecified would discard the classification the caller believed it made.

Demotion consumes declines as a second, much lower-floor rule, and the two rules read disjoint populations: never_triggered counts only offers where nothing was said. The first draft counted every offer, so eight honest no_landing_zone declines tripped the silent-non-use rule and demoted a correct match through the back door — the exact wrong correction the taxonomy exists to prevent, arriving via the other rule.

Two attribution defects found on the way

  1. The advisor recorded only skill on each match, and the CLI has no --skill flag — so every --surface consult wrote skill: null and the entire control arm of all three audit rounds was unattributable to the surface that produced it. surface is now on the event.
  2. detect() enumerated only the declared surface keys. repo-audit:dimension-1 has no table entry — it inherits offload surface-wide — so three audits declining offload there were recorded and never read. A drain that cannot see a surface cannot drain it, and the surfaces most likely to be over-bound are exactly the ones that only inherit.

Reachable from the surfaces that actually decide

CLI decline --capability --experiment --reason --kind --surface, and MCP capability_decline — the callers are skills and bash automations, not Python. The MCP guard is AST-based: a substring check over the handler body was written first and did not discriminate (deleting surface= from the call left the word in the response dict and the break stayed green).

Break→revert, all confirmed to fail without the fix

decline-as-outcome moves the posterior · a blank reason accepted · demotion ignores declines · a triggered capability still demoted · surface attribution dropped · the partition collapses · every kind treated as demotable · the silent rule counting declines · an unknown kind coerced · no_landing_zone demotable · precondition_unmet demotable · surface advertised but not forwarded · reason dropped from the schema · capability_id hardcoded · detect() enumerating declared keys only.

Two of these did not discriminate on the first attempt and were rewritten; both are noted in the code where they live.

Local verdict

python3 verify.py: 365 passed, 82/82 selftests ran, 0 skipped, 4 of 5 gates ok. The 3 pytest failures and the 2 gate failures are pre-existing and machine-local — all five are the evidence-acquisition ledger row (0 references in committed capabilities.py; another session's uncommitted module). Confirmed identical with the change stashed. CI is the verdict.

Contract docs updated (ARCHITECTURE.md, README.md). No diagram change: no stage, component, rail/role classification or role-registry entry moved.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your current included review allowance is based on your included PR review attempts over the past 7 days.

Next review available in: 46 minutes

Limit details: You’ve used the included review currently available. Your 70 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 86bf90c8-27f4-409b-b8bb-ead202e11fdb

📥 Commits

Reviewing files that changed from the base of the PR and between 4f286ac and 6379950.

📒 Files selected for processing (5)
  • ARCHITECTURE.md
  • README.md
  • capability_advisor.py
  • capability_propensity.py
  • mcp_server.py

Comment @coderabbitai help to get the list of available commands.

Tim Stranske and others added 2 commits August 23, 2026 00:20
… with a kind

Two independent audit rounds on 2026-08-23 reached the same conclusion: propensity
carried information exactly once, because most candidates sat at the uninformative
prior. The missing input was not more consults but DECLINES -- a capability declined
on repo-specific grounds looked IDENTICAL in the ledger to one nobody ever considered
(`trig 0, use 0, no 0`). Four of nine decisions in one round and 21 of the offers in
another were reasoned rejections and none was learnable.

A DECLINE IS A THIRD STATE, NOT A NEGATIVE OUTCOME, and the separation is structural
rather than conventional. A decline means the capability did NOT run, so recording it
as an `outcome` would bucket it into `not_useful` -- asserting we tried it and it did
not help, about something that never executed, corrupting the one signal declines exist
to sharpen. So a decline rides on a `match` event (it genuinely WAS offered, which is
the only claim the event type makes) tagged `source=capability_decline`, and
`usefulness()` reads `outcome` events only. There is no code path from a decline to the
posterior. No new event type, no new store: `record_promotion` already carries a
non-match fact on `match` distinguished by `metadata.source`.

`triggered` / `declined` / `not_triggered_silently` now partition the candidate set, so
"rejected on stated grounds", "offered and ignored" and "never considered" are three
different findings. `propensity()` reports the decline count BESIDE the posterior --
"prior, no evidence" and "prior, no evidence, four reasoned rejections" were previously
identical readings.

AND A DECLINE HAS A KIND, because one undifferentiated count licenses the wrong
correction. A third audit round declined 25 offers across six classes with opposite
fixes: `testgen-lane` matched CORRECTLY three times and was structurally impossible (a
read-only audit has no commit target) -- fix: nothing; `offload` was declined at nine
surfaces because it is declared surface-wide and a one-subsystem audit has nothing big
enough to hand off -- fix: a precondition or a narrower declaration; and
`frontend-verifier`, declined on two frontend-less repos, then produced the
second-strongest finding of an audit on a repo that DOES have a display surface -- fix:
evaluate the condition, do not weaken the binding. `demotable` is therefore a property
of the KIND (`DECLINE_KINDS`), declared once: only `wrong_match` and `scope_too_small`
can propose a demotion. An unknown kind is refused rather than coerced.

Demotion consumes declines as a second, much lower-floor rule, and the two rules read
DISJOINT populations: `never_triggered` counts only offers where nothing was said. The
first draft counted every offer, so eight honest `no_landing_zone` declines tripped the
silent-non-use rule and demoted a correct match through the back door -- the exact wrong
correction the taxonomy exists to prevent, arriving via the other rule.

Attribution is now on the event: the advisor records `surface` on each `match`. It
recorded only `skill`, and the CLI has no `--skill` flag, so every `--surface` consult
wrote `skill: null` and the entire control arm of those audit rounds was unattributable
to the surface that produced it.

Reachable from bash (`decline --capability --experiment --reason --kind --surface`) and
from MCP (`capability_decline`), because the surfaces that make these decisions are
skills and lane automations, not Python. The MCP guard is AST-based: a substring check
over the handler body was written first and did NOT discriminate -- deleting `surface=`
from the call left the word in the response dict and the break stayed green.

Break->revert demonstrations, all confirmed to fail without the fix: decline-as-outcome
moves the posterior; a blank reason is accepted; demotion ignores declines; a triggered
capability is still demoted; surface attribution dropped; the partition collapses; every
kind treated as demotable; the silent rule counting declines; an unknown kind coerced;
`no_landing_zone` and `precondition_unmet` declared demotable; `surface` advertised but
not forwarded; `reason` dropped from the schema; `capability_id` hardcoded.

Pre-existing on this machine and unrelated: 3 pytest failures for the `evidence-acquisition`
ledger row (machine-local, 0 references in committed capabilities.py), confirmed
identical with the change stashed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…inding

`detect` enumerated `SURFACE_RECORD_GLOBS | SURFACE_BINDINGS` — the DECLARED keys. But
`repo-audit:dimension-1` has no table entry of its own; it inherits `offload`
surface-wide. So when three independent 2026-08-23 audit rounds each declined `offload`
at that surface with a stated reason, all three were written to the ledger and none was
ever read. A drain that cannot see a surface cannot drain it, and the surfaces most
likely to be OVER-BOUND are exactly the ones that only inherit.

`observed_surfaces()` derives the set from the trials themselves rather than adding a
second list, which would drift from the surfaces that exist and would still omit the
inheriting ones.

The first version of the selftest asserted `observed_surfaces()` and stayed GREEN when
`detect()` was reverted to the declared-keys-only enumeration — a test of the helper
instead of the answer, which is this project's most-repeated testing mistake. It now
asserts what the CALLER receives, and the break fails as required.

Effect on the live ledger, report-only as always: demotion proposals go 2 -> 3, the new
one being `repo-audit:dimension-1 -> offload`, declined in 3 of 3 offers across three
audits, all three attributable to the binding.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@stranske
stranske force-pushed the claude/declines-and-applies-to branch from 806a2fd to 6379950 Compare August 23, 2026 05:21
@stranske
stranske merged commit d90d5de into main Aug 23, 2026
20 checks passed
@stranske
stranske deleted the claude/declines-and-applies-to branch August 23, 2026 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant